Most businesses employ data centers to facilitate online transactions. For example, stock brokerage firms have data centers that provide online brokerage services to users via their client computer systems, including automated placement of orders to buy or sell securities (i.e., stocks or bonds), portfolio tracking, and related market information, news, and services 24 hours a day, seven days a week.
Data centers typically consist of components (e.g., servers, hubs, switches, storage systems, load balancers, etc.) that in combination reply to user requests (e.g., electronic requests to buy or sell stock in a publicly traded company). Replies from data centers may take many forms. For example, replies may take form in web pages containing information (e.g., the price at which stock in a company was purchased or sold) corresponding to respective user requests.
Data centers are measured by the quality of service (QoS) they provide to users. QoS in turn is determined by many factors including reply time, the overall time it takes a data center to reply to a user request. Data centers are processing bandwidth limited. In other words, data centers can process a limited number of user requests in a given period of time. If a data center becomes overloaded with user requests, reply times for the user requests will increase. Long reply times are viewed unfavorably.
Data center components (e.g., servers, switches, disk arrays, etc.) are often manufactured by different vendors. Nonetheless, the components are required to cooperate with each other when responding to user requests from client computer systems. A set of system requests are generated within a data center in response to the data center receiving a user request. These internally generated system requests call on various components within the data center to perform specific tasks. For example, a user request may be received by a web server of a data center. To respond to the user request, the web server may need the services provided by an application server, and accordingly the web server transmits a request for data processing to the application server. To respond to the application processing request, the application server may need to access data stored in a database, and accordingly the application server transmits a request to access data to a database server. The database server may need to access a disk array in order to reply to the application server's request, and accordingly the database server generates a request to access data in the disk array. Eventually, the application server receives a reply from the database server after the database server receives a reply to its request to access data in the disk array. The application server may process data included in the reply from the database server, which leads to the eventual reply to the user request by the web server.
Architectures (i.e., the way components are coupled together to provide data center services) vary from data center to data center. For example, components of data centers may be cobbled together as a distributed n-tier system or as a utility computing system. FIG. 1 illustrates relevant components of an exemplary data center 10 configured as a distributed n-tier system, while FIG. 2 illustrates relevant components of an exemplary utility computing data center 40.
The data center 10 of FIG. 1 includes a load balancer 12 coupled to client computer systems 14 and 16 via a network such as the Internet. The load balancer in turn is coupled to a tier of web servers 20a-20w. User requests are received by the load balancer by client computer systems and subsequently distributed to web servers 20a-20w so that no web server becomes overloaded with user requests relative to another. A tier of application servers 22a-22x are coupled to web servers 20a-20w via a local area network (LAN) 24. Application servers 22a-22x are coupled to a tier of database servers 26a-26y via data access network 30. Database servers 26a-26y execute database management systems that maintain and provide one or more databases for data access by application servers 22a-22x. Lastly, a storage area network (SAN) 32 consisting of switches, routers, and/or hubs (not shown) couples database servers 26a-26y to a tier of storage systems 34a-34z. While it is said that databases store data, in reality the data is stored in memory devices of storage systems 34a-34z. For purposes of explanation, each of storage systems 34a-34z takes form in a disk array, it being understood that the term storage systems should not be limited thereto.
Utility computing is becoming an increasingly popular architecture in data centers. Utility computing is a service provisioning model in which data storage and processing services can be made available on an “as needed basis.” In utility computing, different entities are charged for their actual use of data storage and/or processing services. The exemplary utility computing data center 40 in FIG. 2 includes a processing resource group 42 coupled to storage resource group 44 via network resource group 46. Processing resource group 42 includes a plurality of servers which can be dynamically allocated on demand to perform various processing tasks. For example servers in group 42 can be allocated to execute application programs, database management systems, file systems, etc. Network resource group 46 includes SAN switches, routers (not shown), etc. The components of network resource group can be dynamically allocated to create data paths between components (e.g., disk arrays) of storage resource group 44 and servers of processing resource group 42.
Regardless of how a data center is configured, data center components (i.e., servers, switches, disk arrays, etc.) are required to act in concert in order to reply to user requests from client computer systems. The term execution path will be used herein to refer to a combination of components that collectively process user requests from client computer systems. As will be more fully described below, each component in the execution path may introduce an unanticipated element of delay as a result of being overloaded with requests from other components. The delay introduced by any one or more components of the execution path increases the overall time needed to reply to user requests. It should be noted that data centers may contain more than one execution path.
FIG. 3 illustrates relevant components of an exemplary execution path within a data center such as data center 10 of FIG. 2. From bottom to top, the components of the execution path in FIG. 3 include a disk array 50, database server 52, application server 54, and web server 56. Data paths (not shown) are provided between the components and function to transmit system requests as will be more fully described below.
Disk array 50 contains several hard disks. Each hard disk includes sequentially numbered physical memory blocks for storing data. The hard disks of array 50 may be logically combined by storage software (not shown) to present database server 52 with disk-like storage entities commonly known as virtual disks. Virtual disks typically have better characteristics (e.g., higher storage capacity, greater effective data transfer rates, etc.) than their underlying hard disks. In essence, a virtual disk can be seen as a set of sequentially numbered logical data blocks. Each logical data block is mapped to one or more physical data blocks of one or more underlying hard disks. While it may be said that a logical data block stores data, in reality data of the logical block is stored in a physical memory block mapped thereto by the storage software executing on the disk array. Any request received by disk array 50 to access data of a logical disk block must be translated into one or more requests to access one or more physical memory blocks allocated thereto.
As the value of data sharing becomes apparent, database management systems evolved to provide application-independent means of organizing data so that multiple applications could use the same data and so that the data itself could evolve independently from applications. Database management systems transform numbered data blocks into more business-oriented data items, such as character strings, binary and floating point numbers, and arrays of these objects. The most popular form of database is the relational database which organizes related data items into records and orders sets of like records into tables. Relational database management systems also maintain relationships between records of different types and enforce data consistency roles and transaction constraints. Database server 52 executes a database management system that organizes one or more databases on virtual disks presented by disk array 50. Accordingly, any request to access database data stored on a virtual disk must be translated by the database server 52 into a request to access a virtual disk provided by disk array 50.
Application server 54 processes requests received from web software 56 in accordance with a set of rules or algorithms defined in an application program. Application server 54 often generates requests to access (i.e., read or write) data from one or more databases provided by database server 52 in response to processing requests from web server 56. Web server 56 receives user requests from client computer systems coupled thereto. User requests to run an application are passed to application server 54. Typically, web server 56 sends replies (e.g., web-pages, forms, etc.) to client software (e.g., browser) using HTTP after user requests have been completed in execution path 10.
The components of an executing path, such as the execution path shown in FIG. 3, cooperate to reply to user requests received from client computer systems. When a user request is received from a client computer system, a process is started in an execution path, such as the execution path shown in FIG. 3, which ultimately leads to the generation of a corresponding reply. Cooperation between components will be described with reference to the execution path shown in FIG. 3.
Presume web server 56 receives a user request UR from a client computer system. User requests are stored in a web server queue 57 in the order received from client computer systems until they can be processed by a process executing on one or more processors in web server 56. User requests are output from the web server queue 57 for processing in the order they are received therein. Web server 56 may generate one or more application processing requests when it receives and processes a user request. For purposes of explanation only, it will be presumed that web server 56 generates a single application processing request in response to receiving and processing a user request. As such, web server 56 generates an application processing request APR in response to processing the user request UR.
Application server 54 receives the application processing request APR from web server 56. Application server 54 includes an application server queue 55 that stores application processing requests in the order received from web server 56 or other web servers (not shown in FIG. 3). Application processing requests are outputted from queue 55 for processing in the order received by an application program executing on one or more processors. Application server 54, in response to processing each application processing request, may generate one or more requests to read data from or write data to one or more databases managed by database server 52. For purposes of explanation only, it will be presumed that application server 54 generates a single database access request in response to receiving and processing an application processing request. Thus, application server 54 generates a database access request DBAR in response to processing the application processing request APR.
Database server 52 receives the database access request DBAR from application server 54. Database server 52, like application server 54, includes a queue 53 that stores database access requests in the order received from application server 54 or other application servers (not shown in FIG. 3). Database access requests are outputted from queue 55 for processing in the order received by database management system executing on one or more processors. Database server 52, in response to processing each database access request, generates one or more requests to access one or more virtual disks provided by disk array 50. For purposes of explanation only, it will be presumed that database server 52 generates a virtual disk access request in response to receiving and processing a database access request. Accordingly, database server 52 generates virtual disk access request VDAR in response to processing the database access request DBAR.
Disk array 50 receives the virtual disk access request VDAR from database server 52. Disk array 50 includes a queue 51 that stores virtual disk access requests in the order received from database server 52 or other database servers (not shown in FIG. 3). Virtual disk access requests are outputted from queue 51 for processing in the order received by disk virtualizing software executing on one or more processors. Virtualizing software executing on disk array 50, in response to processing each virtual disk access request, generates one or more requests to access data in one or more hard disks of disk array 50. For purposes of explanation only, it will be presumed that disk array 50 generates a single hard disk access request in response to receiving and processing a single virtual disk access request. Accordingly, disk array 50 generates a hard disk access request HDAR in response to processing the virtual disk access request VDAR. Disk array 50 completes hard disk access requests by reading data from or writing data to one or more hard disks.
Processes executing on components 50-56 generate replies corresponding to the requests they receive. For example, after the hard disk access request HDAR has completed (i.e., data is read from or written to a hard disk of disk array 50), the disk virtualizing system of disk array 50 generates a virtual disk access reply VDARR indicating that the corresponding virtual disk access request VDAR has been completed. VDARR may include, for example, information read from a hard disk or an acknowledgement that a write operation has been completed. The database management system of database server 52 generates a database access request reply DBARR in response to receiving and processing virtual disk access request reply VDARR. Database access request reply DBARR corresponds to DBAR and is provided to application server 54. The application program of application server 54 generates an application processing request reply APRR in response to receiving and processing the database access request reply DBARR. Any data contained in the replies from database server 52 may be processed in accordance with the rules or algorithm defined in the application executing on server 54. The application processing request reply APRR corresponds to the application processing request APR and is transmitted to web server 56. The process executing on web server 56 generates a user request reply URR in response to receiving and processing the application processing request reply APRR. The user request reply URR corresponds to the original user request UR and is transmitted to the client computer system that generated the user request UR.
Each of the components is processing bandwidth limited. In other words, each of the components in the data path shown in FIG. 3 can process a limited number of requests in a given period of time. The request queues (e.g., application server queue 55) store requests in the order received until they can be processed. There is a delay between the time a new request (e.g., a new application processing request APR) is received in a queue and the time when the new request is outputted for processing. The delay in the queue for each newly received request depends on the number of prior requests pending in the queue. If the queue stores a large number of prior requests, the time between the new request is received in the queue and the time when the new request is outputted for processing will be relatively large.
At any given time, it is possible that the queue of a component in execution path of FIG. 3 may store an excessively large number of requests. For example, queue 55 of application server 54 may store an unusually large number of application processing requests received from web server 56 or other web servers (not shown). Because the queue stores a large number of application processing requests, the processing of any new request received by the queue will be delayed substantially since requests are processed in the order they are received in the queue. The delay caused by a crowded queue will, in turn, delay replies to several user requests received by web server 56 while the application processing request queue is overcrowded.
Increased reply times may have negative implications. To illustrate, presume the execution path illustrated in FIG. 3 is configured to process user requests to buy or sell shares of stock in publicly traded companies. Presume also that the execution path receives user requests to buy or sell stock from many different client computer systems. Some of these user requests may include orders to buy or sell a small number of shares and accordingly involve a small amount of money, while other user requests include orders to buy or sell a large number of shares and involve a large amount of money. Suppose further that application server queue 55 stores more application processing requests than can be processed by the application program without significant delay. As a result, reply times for user requests to buy or sell stock will be delayed significantly while the application server queue 55 is crowded with pending requests. Indeed, the delay introduced by the application server queue 55 of application server 54 may be so long that sellers who initiate user requests may end up losing a large amount of money if their user requests to sell a large number or shares are delayed while the market price in their stocks rapidly decreases.